

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>RGW 动态的桶索引重分片 &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="RGW 对多因子认证的支持情况" href="../mfa/" />
    <link rel="prev" title="桶策略" href="../bucketpolicy/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../">Ceph 对象网关</a> &raquo;</li>
        
      <li>RGW 动态的桶索引重分片</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/radosgw/dynamicresharding.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../">
          

          
            
            <img src="../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 对象网关</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../frontends/">HTTP 前端</a></li>
<li class="toctree-l2"><a class="reference internal" href="../placement/">存储池归置与存储类</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multisite/">多站配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multisite-sync-policy/">多站同步策略配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../pools/">存储池的配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../config-ref/">配置参考</a></li>
<li class="toctree-l2"><a class="reference internal" href="../admin/">管理指南</a></li>
<li class="toctree-l2"><a class="reference internal" href="../s3/">S3 API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rgw-cache/">Data caching and CDN</a></li>
<li class="toctree-l2"><a class="reference internal" href="../swift/">Swift API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../adminops/">管理操作 API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/">Python 接口</a></li>
<li class="toctree-l2"><a class="reference internal" href="../nfs/">通过 NFS 导出</a></li>
<li class="toctree-l2"><a class="reference internal" href="../keystone/">与 OpenStack Keystone 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../barbican/">与 OpenStack Barbican 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../vault/">与 HashiCorp Vault 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kmip/">KMIP Integration</a></li>
<li class="toctree-l2"><a class="reference internal" href="../opa/">与 Open Policy Agent 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multitenancy/">多租户</a></li>
<li class="toctree-l2"><a class="reference internal" href="../compression/">压缩</a></li>
<li class="toctree-l2"><a class="reference internal" href="../ldap-auth/">LDAP 认证</a></li>
<li class="toctree-l2"><a class="reference internal" href="../encryption/">服务器端加密</a></li>
<li class="toctree-l2"><a class="reference internal" href="../bucketpolicy/">桶策略</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">动态的桶索引重分片</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#id1">多站</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id2">配置选项</a></li>
<li class="toctree-l3"><a class="reference internal" href="#id3">管理命令</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#id4">把一个桶加进重分片队列</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id5">罗列重分片队列</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id6">处理重分片队列里的任务</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id7">桶重分片状态</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id8">取消挂着的桶重分片操作</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id9">手动执行即时桶重分片</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#id10">故障排除</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#id11">掉队例程管理</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id12">生命周期修复</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id13">对象逾期管理器修复</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../mfa/">多因子认证</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sync-modules/">同步模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../notifications/">Bucket Notifications</a></li>
<li class="toctree-l2"><a class="reference internal" href="../layout/">RADOS 中的数据布局</a></li>
<li class="toctree-l2"><a class="reference internal" href="../STS/">STS</a></li>
<li class="toctree-l2"><a class="reference internal" href="../STSLite/">STS Lite</a></li>
<li class="toctree-l2"><a class="reference internal" href="../keycloak/">Keycloak</a></li>
<li class="toctree-l2"><a class="reference internal" href="../role/">Role</a></li>
<li class="toctree-l2"><a class="reference internal" href="../session-tags/">Session Tags</a></li>
<li class="toctree-l2"><a class="reference internal" href="../orphans/">Orphan List and Associated Tooliing</a></li>
<li class="toctree-l2"><a class="reference internal" href="../oidc/">OpenID Connect Provider</a></li>
<li class="toctree-l2"><a class="reference internal" href="../troubleshooting/">故障排除</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../man/8/radosgw/">radosgw 手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../man/8/radosgw-admin/">radosgw-admin 手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../qat-accel/">使用 QAT 为加密和压缩提速</a></li>
<li class="toctree-l2"><a class="reference internal" href="../s3select/">S3-select</a></li>
<li class="toctree-l2"><a class="reference internal" href="../lua-scripting/">Lua Scripting</a></li>
<li class="toctree-l2"><a class="reference internal" href="../d3n_datacache/">D3N Data Cache</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cloud-transition/">Cloud Transition</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="rgw">
<span id="rgw-dynamic-bucket-index-resharding"></span><h1>RGW 动态的桶索引重分片<a class="headerlink" href="#rgw" title="Permalink to this headline">¶</a></h1>
<div class="versionadded">
<p><span class="versionmodified added">New in version Luminous.</span></p>
</div>
<p>巨大的桶索引会导致性能问题，为了解决此问题，我们引进了桶索引分片（机制）。
到 Luminous 版为止，更改桶的分片数量（重分片， resharding ）需要离线操作。
从 Luminous 起才支持桶的在线重分片。</p>
<p>Each bucket index shard can handle its entries efficiently up until
reaching a certain threshold number of entries. If this threshold is
exceeded the system can suffer from performance issues. The dynamic
resharding feature detects this situation and automatically increases
the number of shards used by the bucket index, resulting in a
reduction of the number of entries in each bucket index shard. This
process is transparent to the user. Write I/Os to the target bucket
are blocked and read I/Os are not during resharding process.</p>
<p>By default dynamic bucket index resharding can only increase the
number of bucket index shards to 1999, although this upper-bound is a
configuration parameter (see Configuration below). When
possible, the process chooses a prime number of bucket index shards to
spread the number of bucket index entries across the bucket index
shards more evenly.</p>
<p>The detection process runs in a background process that periodically
scans all the buckets. A bucket that requires resharding is added to
the resharding queue and will be scheduled to be resharded later. The
reshard thread runs in the background and execute the scheduled
resharding tasks, one at a time.</p>
<div class="section" id="id1">
<h2>多站<a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h2>
<p>动态重分片功能还不能在多站环境下使用。</p>
</div>
<div class="section" id="id2">
<h2>配置选项<a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h2>
<p>启用、禁用动态的桶索引重分片功能：</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">rgw_dynamic_resharding</span></code>:  true/false, 默认： true.</p></li>
</ul>
<p>用于控制重分片进程的配置选项：</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">rgw_max_objs_per_shard</span></code>: 每个桶索引分片内的对象数最大多少时就触发重分片，默认： 100000 对象。</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">rgw_max_dynamic_shards</span></code>: 动态桶索引重分片可以达到的最大分片数，默认： 1999</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">rgw_reshard_bucket_lock_duration</span></code>: 重分片时锁住桶对象的时长，按秒算，默认： 360 秒（即6分钟）</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">rgw_reshard_thread_interval</span></code>: 两轮重分片队列处理的最大间隔时间，按秒算，默认： 600 秒（即10分钟）</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">rgw_reshard_num_logs</span></code>: 重分片队列的分片数量，默认： 16</p></li>
</ul>
</div>
<div class="section" id="id3">
<h2>管理命令<a class="headerlink" href="#id3" title="Permalink to this headline">¶</a></h2>
<div class="section" id="id4">
<h3>把一个桶加进重分片队列<a class="headerlink" href="#id4" title="Permalink to this headline">¶</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard add --bucket &lt;bucket_name&gt; --num-shards &lt;new number of shards&gt;</span>
</pre></div>
</div>
</div>
<div class="section" id="id5">
<h3>罗列重分片队列<a class="headerlink" href="#id5" title="Permalink to this headline">¶</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard list</span>
</pre></div>
</div>
</div>
<div class="section" id="id6">
<h3>处理重分片队列里的任务<a class="headerlink" href="#id6" title="Permalink to this headline">¶</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard process</span>
</pre></div>
</div>
</div>
<div class="section" id="id7">
<h3>桶重分片状态<a class="headerlink" href="#id7" title="Permalink to this headline">¶</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard status --bucket &lt;bucket_name&gt;</span>
</pre></div>
</div>
<p>以下输出是每个分片里有 3 个对象的 json 数组（reshard_status, new_bucket_instance_id, num_shards）。</p>
<p>例如，在动态重分片的不同阶段，输出分别如下：</p>
<p><code class="docutils literal notranslate"><span class="pre">1.</span> <span class="pre">重分片之前：</span></code></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span>
  <span class="p">{</span>
      <span class="s2">&quot;reshard_status&quot;</span><span class="p">:</span> <span class="s2">&quot;not-resharding&quot;</span><span class="p">,</span>
      <span class="s2">&quot;new_bucket_instance_id&quot;</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="p">,</span>
      <span class="s2">&quot;num_shards&quot;</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span>
  <span class="p">}</span>
<span class="p">]</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">2.</span> <span class="pre">重分片期间：</span></code></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span>
  <span class="p">{</span>
      <span class="s2">&quot;reshard_status&quot;</span><span class="p">:</span> <span class="s2">&quot;in-progress&quot;</span><span class="p">,</span>
      <span class="s2">&quot;new_bucket_instance_id&quot;</span><span class="p">:</span> <span class="s2">&quot;1179f470-2ebf-4630-8ec3-c9922da887fd.8652.1&quot;</span><span class="p">,</span>
      <span class="s2">&quot;num_shards&quot;</span><span class="p">:</span> <span class="mi">2</span>
  <span class="p">},</span>
  <span class="p">{</span>
      <span class="s2">&quot;reshard_status&quot;</span><span class="p">:</span> <span class="s2">&quot;in-progress&quot;</span><span class="p">,</span>
      <span class="s2">&quot;new_bucket_instance_id&quot;</span><span class="p">:</span> <span class="s2">&quot;1179f470-2ebf-4630-8ec3-c9922da887fd.8652.1&quot;</span><span class="p">,</span>
      <span class="s2">&quot;num_shards&quot;</span><span class="p">:</span> <span class="mi">2</span>
  <span class="p">}</span>
<span class="p">]</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">3,</span> <span class="pre">重分片完成之后：</span></code></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span>
  <span class="p">{</span>
      <span class="s2">&quot;reshard_status&quot;</span><span class="p">:</span> <span class="s2">&quot;not-resharding&quot;</span><span class="p">,</span>
      <span class="s2">&quot;new_bucket_instance_id&quot;</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="p">,</span>
      <span class="s2">&quot;num_shards&quot;</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span>
  <span class="p">},</span>
  <span class="p">{</span>
      <span class="s2">&quot;reshard_status&quot;</span><span class="p">:</span> <span class="s2">&quot;not-resharding&quot;</span><span class="p">,</span>
      <span class="s2">&quot;new_bucket_instance_id&quot;</span><span class="p">:</span> <span class="s2">&quot;&quot;</span><span class="p">,</span>
      <span class="s2">&quot;num_shards&quot;</span><span class="p">:</span> <span class="o">-</span><span class="mi">1</span>
  <span class="p">}</span>
<span class="p">]</span>
</pre></div>
</div>
</div>
<div class="section" id="id8">
<h3>取消挂着的桶重分片操作<a class="headerlink" href="#id8" title="Permalink to this headline">¶</a></h3>
<p>注意：正在进行着的桶重分片操作无法取消。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard cancel --bucket &lt;bucket_name&gt;</span>
</pre></div>
</div>
</div>
<div class="section" id="id9">
<h3>手动执行即时桶重分片<a class="headerlink" href="#id9" title="Permalink to this headline">¶</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin bucket reshard --bucket &lt;bucket_name&gt; --num-shards &lt;new number of shards&gt;</span>
</pre></div>
</div>
<p>When choosing a number of shards, the administrator should keep a
number of items in mind. Ideally the administrator is aiming for no
more than 100000 entries per shard, now and through some future point
in time.</p>
<p>Additionally, bucket index shards that are prime numbers tend to work
better in evenly distributing bucket index entries across the
shards. For example, 7001 bucket index shards is better than 7000
since the former is prime. A variety of web sites have lists of prime
numbers; search for “list of prime numbers” withy your favorite web
search engine to locate some web sites.</p>
</div>
</div>
<div class="section" id="id10">
<h2>故障排除<a class="headerlink" href="#id10" title="Permalink to this headline">¶</a></h2>
<p>Clusters prior to Luminous 12.2.11 and Mimic 13.2.5 left behind stale bucket
instance entries, which were not automatically cleaned up. The issue also affected
LifeCycle policies, which were not applied to resharded buckets anymore. Both of
these issues can be worked around using a couple of radosgw-admin commands.</p>
<div class="section" id="id11">
<h3>掉队例程管理<a class="headerlink" href="#id11" title="Permalink to this headline">¶</a></h3>
<p>罗列出集群里已经准备好、可以被清理掉的过时例程。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard stale-instances list</span>
</pre></div>
</div>
<p>清理掉集群里的过时例程。注意：
只能在单站点集群上进行这种例程清理。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin reshard stale-instances rm</span>
</pre></div>
</div>
</div>
<div class="section" id="id12">
<h3>生命周期修复<a class="headerlink" href="#id12" title="Permalink to this headline">¶</a></h3>
<p>For clusters that had resharded instances, it is highly likely that the old
lifecycle processes would have flagged and deleted lifecycle processing as the
bucket instance changed during a reshard. While this is fixed for newer clusters
(from Mimic 13.2.6 and Luminous 12.2.12), older buckets that had lifecycle policies and
that have undergone resharding will have to be manually fixed.</p>
<p>完成修复的命令是：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin lc reshard fix --bucket {bucketname}</span>
</pre></div>
</div>
<p>As a convenience wrapper, if the <code class="docutils literal notranslate"><span class="pre">--bucket</span></code> argument is dropped then this
command will try and fix lifecycle policies for all the buckets in the cluster.</p>
</div>
<div class="section" id="id13">
<h3>对象逾期管理器修复<a class="headerlink" href="#id13" title="Permalink to this headline">¶</a></h3>
<p>Objects subject to Swift object expiration on older clusters may have
been dropped from the log pool and never deleted after the bucket was
resharded. This would happen if their expiration time was before the
cluster was upgraded, but if their expiration was after the upgrade
the objects would be correctly handled. To manage these expire-stale
objects, radosgw-admin provides two subcommands.</p>
<p>罗列：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin objects expire-stale list --bucket {bucketname}</span>
</pre></div>
</div>
<p>Displays a list of object names and expiration times in JSON format.</p>
<p>删除：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># radosgw-admin objects expire-stale rm --bucket {bucketname}</span>
</pre></div>
</div>
<p>它会初始化这些对象的删除、以 JSON 格式显示对象名列表、过期时间、和删除状态。</p>
</div>
</div>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="../mfa/" class="btn btn-neutral float-right" title="RGW 对多因子认证的支持情况" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../bucketpolicy/" class="btn btn-neutral float-left" title="桶策略" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>